This document provides a summary of the change-point analysis for Blastomycosis (blasto).
The following section summarizes change-point analysis for trends in the number of visits before diagnosis using the standard piecewise-modeling approach to find the change-point. Specifically, we evaluate 4 peicewise models with linear, quadratic, cubic and exponential trends. The change-point is found by iterating over different change-points and selecting the best fitting model based on AIC.
| label | Change Point | Pred. Bound CP | CP # Miss | PB CP # Miss |
|---|---|---|---|---|
| Piecewise lm w/ periodicity | 63 | 71 | 6350.29 | 6477.83 |
| Piecewise quad w/ periodicity | 77 | 51 | 4986.88 | 4786.87 |
| Piecewise cubic w/ periodicity | 84 | 51 | 3880.73 | 3767.81 |
| Piecewise exp w/ periodicity | 49 | 51 | 3675.31 | 3708.80 |
| Piecewise lm | 63 | 67 | 6350.52 | 6399.20 |
| Piecewise quad | 77 | 51 | 4988.12 | 4737.32 |
| Piecewise cubic | 84 | 50 | 3939.88 | 3757.99 |
| Piecewise exp | 49 | 51 | 3670.72 | 3696.93 |
The following figure depicts the optimal change-point for each method
with periodicity.
The following figure depicts the optimal change-point for each method
without periodicity.
| weeks | days | RMSE | N Miss Visits | RMSE | N Miss Visits | RMSE | N Miss Visits | RMSE | N Miss Visits |
|---|---|---|---|---|---|---|---|---|---|
| 1 | 7 | 33.388 | 1082 | 21.005 | 655 | 14.327 | 344 | 18.069 | 160 |
| 2 | 14 | 27.829 | 2068 | 17.881 | 1317 | 13.075 | 757 | 17.004 | 570 |
| 3 | 21 | 23.823 | 2925 | 15.907 | 1921 | 12.377 | 1152 | 16.048 | 1064 |
| 4 | 28 | 20.155 | 3751 | 13.993 | 2578 | 11.699 | 1682 | 14.768 | 1721 |
| 5 | 35 | 16.403 | 4531 | 11.754 | 3297 | 10.545 | 2437 | 12.974 | 2485 |
| 6 | 42 | 14.26 | 5126 | 10.775 | 3823 | 10.063 | 2986 | 12.762 | 3050 |
| 7 | 49 | 12.246 | 5690 | 9.684 | 4409 | 9.456 | 3792 | 12.116 | 3675 |
| 8 | 56 | 11.594 | 6040 | 9.547 | 4680 | 9.416 | 4105 | 12.789 | 3984 |
| 9 | 63 | 11.345 | 6350 | 9.472 | 4909 | 9.407 | 4382 | 13.195 | 4258 |
| 10 | 70 | 11.443 | 6600 | 9.443 | 5045 | 9.393 | 4495 | 13.521 | 4454 |
| 11 | 77 | 11.838 | 6722 | 9.423 | 4987 | 9.35 | 4176 | 13.822 | 4484 |
| 12 | 84 | 12.603 | 6922 | 9.438 | 4997 | 9.338 | 3881 | 13.912 | 4614 |
| 13 | 91 | 13.822 | 7216 | 9.455 | 5212 | 9.341 | 4069 | 13.845 | 4927 |
| 14 | 98 | 14.887 | 7467 | 9.462 | 5382 | 9.406 | 4197 | 13.79 | 5202 |
| 15 | 105 | 15.88 | 7657 | 9.524 | 5405 | 9.466 | 3860 | 13.782 | 5368 |
| 16 | 112 | 17.007 | 7949 | 9.555 | 5667 | 9.48 | 4118 | 13.742 | 5729 |
| 17 | 119 | 17.989 | 8220 | 9.729 | 5883 | 9.532 | 4225 | 13.719 | 6058 |
| 18 | 126 | 18.985 | 8524 | 9.912 | 6318 | 9.538 | 5171 | 13.689 | 6506 |
| 19 | 133 | 19.876 | 8817 | 10.213 | 6711 | 9.546 | 6121 | 13.679 | 6915 |
| 20 | 140 | 20.798 | 9206 | 10.451 | 7628 | 9.451 | 9695 | 13.656 | 7564 |
| 21 | 147 | 21.407 | 9310 | 11.146 | 7599 | 9.496 | 10181 | 13.658 | 7662 |
| 22 | 154 | 21.972 | 9312 | 11.815 | 7213 | 9.481 | 9403 | 13.652 | 7583 |
| 23 | 161 | 22.535 | 9253 | 12.385 | 6420 | 9.45 | 6230 | 13.641 | 7359 |
| 24 | 168 | 23.403 | 9518 | 12.633 | 6822 | 9.634 | 8122 | 13.722 | 7772 |
| 25 | 175 | 24.246 | 9852 | 12.907 | 7664 | 9.759 | 13353 | 13.816 | 8338 |
| 26 | 182 | 24.961 | 10082 | 13.317 | 8219 | 9.815 | 18960 | 13.888 | 8717 |
| 27 | 189 | 25.588 | 10179 | 13.809 | 8191 | 9.874 | 23016 | 13.945 | 8842 |
| 28 | 196 | 26.149 | 10106 | 14.327 | 7022 | 9.97 | 20098 | 13.987 | 8602 |
| 29 | 203 | 26.682 | 9907 | 14.817 | 4979 | 10.058 | 8077 | 14.021 | 8098 |
| 30 | 210 | 27.356 | 10004 | 15.163 | 4293 | 10.275 | 3217 | 14.147 | 8191 |
| 31 | 217 | 27.994 | 10029 | 15.526 | 3260 | 10.458 | 5 | 14.273 | 8141 |
| 32 | 224 | 28.606 | 10008 | 15.897 | 2057 | 10.593 | 0 | 14.403 | 7988 |
| 33 | 231 | 29.288 | 10288 | 16.197 | 1406 | 10.805 | 0 | 14.605 | 8414 |
| 34 | 238 | 30.049 | 11224 | 16.424 | 2507 | 11.136 | 5 | 14.89 | 10194 |
| 35 | 245 | 30.695 | 12135 | 16.753 | 4175 | 11.327 | 5 | 15.114 | 11949 |
| 36 | 252 | 31.235 | 12830 | 17.201 | 6423 | 11.431 | 0 | 15.28 | 13307 |
| 37 | 259 | 31.778 | 13971 | 17.604 | 15282 | 11.58 | 19 | 15.46 | 15566 |
| 38 | 266 | 32.291 | 15457 | 17.997 | 33371 | 11.721 | 116 | 15.629 | 18585 |
| 39 | 273 | 32.643 | 15485 | 18.644 | 43114 | 11.767 | 208 | 15.703 | 18756 |
| 40 | 280 | 32.981 | 14832 | 19.257 | 47503 | 11.856 | 59 | 15.771 | 17550 |
| 41 | 287 | 33.46 | 16165 | 19.59 | 85646 | 12.091 | 247006 | 15.957 | 20332 |
| 42 | 294 | 33.786 | 14412 | 20.15 | 92350 | 12.242 | 418886 | 16.029 | 16938 |
The following figure depicts model performance, in terms of RMSE,
across various change-points for each of the methods evaluated:
The following figure depicts the optimal linear model along with the
4 other nearest change-points on either side of the optimal change-point
The following figure depicts the optimal quadratic model along with
the 4 other nearest change-points on either side of the optimal
change-point
The following figure depicts the optimal cubic model along with the 4
other nearest change-points on either side of the optimal change-point
The following figure depicts the optimal exponential model along with
the 4 other nearest change-points on either side of the optimal
change-point
This section summarizes results using counts of SSD-related visits.
The following figure depicts the in-sample and out-of-sample performance (MSE) of various bounds on the opportunity window and different trends.
The following table depicts the top 10 specifications based on either aggregate or k-fold out-of-sample performance:
| rank | Weeks | Days | Model | MSE | Weeks | Days | Model | MSE |
|---|---|---|---|---|---|---|---|---|
| 1 | 7 | 49 | Cubic | 113.56 | 7 | 49 | Cubic | 191.86 |
| 2 | 8 | 56 | Cubic | 114.15 | 8 | 56 | Cubic | 192.30 |
| 3 | 9 | 63 | Cubic | 114.80 | 9 | 63 | Cubic | 193.06 |
| 4 | 9 | 63 | Quadratic | 115.09 | 9 | 63 | Quadratic | 193.29 |
| 5 | 11 | 77 | Cubic | 115.35 | 8 | 56 | Quadratic | 193.42 |
| 6 | 8 | 56 | Quadratic | 115.37 | 11 | 77 | Cubic | 193.59 |
| 7 | 10 | 70 | Quadratic | 115.57 | 10 | 70 | Quadratic | 193.68 |
| 8 | 10 | 70 | Cubic | 115.65 | 10 | 70 | Cubic | 193.85 |
| 9 | 12 | 84 | Cubic | 115.85 | 12 | 84 | Cubic | 194.13 |
| 10 | 13 | 91 | Cubic | 116.08 | 13 | 91 | Cubic | 194.36 |
The following figure depicts the observed and expected trend for the top 9 models based on 99-fold out-of-sample performance:
The following table summarizes the out-of-sample and 99-fold performance (RMSE) along with the implied number of missed opportunities for each method across the different change-points evaluated
| weeks | days | Out-of-sample RMSE | K-fold RMSE | N Miss Visits | Out-of-sample RMSE | K-fold RMSE | N Miss Visits | Out-of-sample RMSE | K-fold RMSE | N Miss Visits |
|---|---|---|---|---|---|---|---|---|---|---|
| 1 | 7 | 1037.57 | 1102.19 | 1096 | 415.36 | 488.33 | 678 | 204.95 | 280.41 | 372 |
| 2 | 14 | 734.94 | 801.45 | 2090 | 312.11 | 385.01 | 1351 | 178.22 | 253.15 | 800 |
| 3 | 21 | 539.62 | 614.48 | 2957 | 249.97 | 327.28 | 1973 | 163.9 | 241.22 | 1223 |
| 4 | 28 | 384.62 | 461.99 | 3796 | 196 | 274.38 | 2660 | 147.75 | 225.6 | 1814 |
| 5 | 35 | 268.54 | 349.56 | 4571 | 150.99 | 231.57 | 3365 | 128.85 | 208.38 | 2549 |
| 6 | 42 | 208.43 | 288.52 | 5173 | 131.15 | 211.25 | 3910 | 121.12 | 200.59 | 3150 |
| 7 | 49 | 165.86 | 243.34 | 5718 | 116.48 | 194.86 | 4448 | 113.56 | 191.86 | 3866 |
| 8 | 56 | 152.35 | 229.12 | 6068 | 115.37 | 193.42 | 4712 | 114.15 | 192.3 | 4173 |
| 9 | 63 | 143.23 | 220.44 | 6382 | 115.09 | 193.29 | 4945 | 114.8 | 193.06 | 4473 |
| 10 | 70 | 137.91 | 214.88 | 6644 | 115.57 | 193.68 | 5103 | 115.65 | 193.85 | 4661 |
| 11 | 77 | 138.87 | 216.09 | 6734 | 116.29 | 194.41 | 4898 | 115.35 | 193.59 | 4001 |
| 12 | 84 | 135.95 | 213.6 | 6957 | 116.81 | 195.03 | 4956 | 115.85 | 194.13 | 3851 |
| 13 | 91 | 131.02 | 208.87 | 7260 | 116.47 | 194.72 | 5202 | 116.08 | 194.36 | 4117 |
| 14 | 98 | 129.82 | 207.98 | 7493 | 117.92 | 196.18 | 5308 | 117.69 | 195.9 | 4069 |
The following figure depicts the optimal linear model (based on
99-fold performance) along with the 4 other nearest change-points on
either side of the optimal change-point
The following figure depicts the optimal quadratic model (based on
99-fold performance) along with the 4 other nearest change-points on
either side of the optimal change-point
The following figure depicts the optimal cubic model (based on
99-fold performance) along with the 4 other nearest change-points on
either side of the optimal change-point